Toward a Hybrid Intelligence System in Customer Service: Collaborative Learning of Human and AI

Hybrid intelligence systems (HIS) enable human users and Artificial Intelligence (AI) to collaborate in activities complementing each other. They particularly allow the combination of human-in-the-loop and computer-in-the-loop learning ensuring a hybrid collaborative learning cycle. To design such a HIS, we implemented a prototype based on formulated design principles (DPs) to teach and learn from its human user while collaborating on a task. For implementation and evaluation, we selected a customer service use case as a top domain of research on AI applications. The prototype was evaluated with 31 expert and 30 novice customer service employees of an organization. We found that the prototype following the DPs successfully contributed to positive learning effects as well as a high continuance intention to use. The measured levels of satisfaction and continuance intention to use provide promising results to reuse our DPs and further develop our prototype for hybrid collaborative learning

Integration of AI into Customer Service: A Taxonomy to Inform Design Decisions

Artificial Intelligence (AI) is increasingly deployed in customer service for various service delivery tasks. Research and practice alike have extensively dealt with the use, benefits, and effects of AI solutions in customer service contexts. Nevertheless, knowledge on AI integration is dispersed and unsystematized. This paper addresses this gap by presenting a taxonomy to inform design decisions for the integration of AI into customer service with five meta-dimensions, 12 dimensions, and 32 characteristics. Through a rigorous and systematic development process comprising multiple iterations and evaluation episodes, state-of-the-art AI solutions from practice and the current state of knowledge from research were systematized to classify AI use cases. Thus, we contribute with systemized design knowledge to, both, the theoretical knowledge base as well as to practice for application. Eventually, we disclose future research avenues addressing certain meta-dimensions as well as the extension of the taxonomy itself

WHAT SHOULD AI KNOW? INFORMATION DISCLOSURE IN HUMAN-AI COLLABORATION

AI-assisted Design Thinking shows great potential for supporting collaborative creative work. To foster creative thinking processes within teams with individualized suggestions, AI has to rely on data provided by the teams. As a prerequisite, team members need to weigh their disclosure preferences against the potential benefits of AI when disclosing information. To shed light on these decisions, we identify relevant information such as emotional states or discussion arguments that design thinking teams could provide to AI to enjoy the benefits of its support. Using the privacy calculus as theoretical lens, we draft a research design to analyze user preferences for disclosing different information relevant to the service bundles that AI provides for respective information. We make explorative contributions to the body of knowledge in terms of AI use and its corresponding information disclosure. The findings are relevant for practice as they guide the design of AI that fosters information disclosure

Conquering the Challenge of Continuous Business Model Improvement - Design of a Repeatable Process

In an atmosphere of rapidly changing business environments and intense competition, adequate and timely business models are crucial for companies. Current research mainly focuses on business model development that often neglects the legacy of established companies. The paper at hand addresses this research gap by a process design which allows established companies to rethink, improve, and continually innovate their business models. Following a design science research approach, require- ments for improving business models are identified by the analysis of existing literature and by expert interviews. Collaboration Engineering and a multilevel evaluation are applied to create a continuous and implementable process design for business model improvement – including specific activities, instructions, and tools. The process design represents a nascent design theory in form of an ‘‘invention’’ type of knowledge contribution. Moreover, going beyond existing literature, the importance of col- laboration between participants in a business model improvement project is highlighted. From a practical per- spective, the developed process design enables companies for continuous and recurring business model improvement without the ongoing support of professional moderators or consultants

Augmented Facilitation: Designing a multi-modal Conversational Agent for Group Ideation

Human facilitators face the challenge to structure and collect relevant insights from collaborative creative work sessions, which can suffer if they face a high workload. Hence, for effective value co-creation in organizational ideation we suggest an facilitation augmentation with a conversational agent (CA). CAs have the ability to support respective collaborative work by documenting and analyzing unstructured data. Following the design science research paradigm, and based on the literature about facilitation and human-AI collaboration, we derive design principles to develop a CA prototype that collects ideas from a group ideation session and displays them back in a structured (multi-modal) manner. We evaluate the CA by conducting four focus groups. Key findings show that the CA successfully distills and enriches information. Our study contributes to understanding the role of CA in augmenting facilitation and it provides guidance for practice on how to integrate these technologies in group meetings

A User-centric Taxonomy for Conversational Generative Language Models

Conversational generative language models (GLMs) like ChatGPT are being rapidly adopted. Previous research on non-conversational GLMs showed that formulating prompts is critical for receiving good outputs. However, it is unclear how conversational GLMs are used when solving complex problems that require multi-step interactions. This paper addresses this research gap based on findings from a large participant event we conducted, where ChatGPT was iteratively and in a multi-step manner used while solving a complex problem. We derived a taxonomy of prompting behavior employed for solving complex problems as well as archetypes. While the taxonomy provides common knowledge on GLMs usage based on analyzed input-prompts, the different archetypes facilitate the classification of operators according to their usage. With both we provide exploratory knowledge and a foundation for design science research endeavors, which can be referred to, enabling further research and development of prompt engineering, prompting tactics, and prompting strategies on common ground

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

(Re)Designing IT Support: How Embedded and Conversational AI Can Augment Technical Support Work

Striving for operational efficiency and cost-effectiveness, companies increasingly deploy artificial intelligence (AI). This trend also incrementally permeates service-related work in technical support. As current narrow AI cannot fully substitute service employees and greater effects are achieved with hybrid service delivery, adapted work settings are required. Based on a qualitative field study with a socio-technical approach, this research provides current problem scenarios in IT support and a support process redesign by integrating conversational and embedded AI. The study contributes evaluated insights about current work processes, work-related issues, and a hybrid IT support process that introduces substitution and augmentation of human tasks to improve service delivery

The Collaboration of Crowd Workers

Crowdsourcing is used for problem solving in different domains. A promising key to optimal solutions is collaboration among crowd workers. However, due to the distributed and asynchronous nature of crowd work, often with a large number of heterogeneous, anonymous and varying workers, crowd collaboration support is challenging. Thus, platform providers and crowdsourcers still struggle with or refrain from making full use of the potential of collaboration. The current state of research on this field explores this topic mostly for a specific domain, such as idea contests. This paper widens this scope and aims to validate a general process design framework for collaboration in crowdsourcing across various domains in an ongoing design science research project. To achieve this, we analyze current projects on crowdsourcing platforms with a conceptual process structure and a corresponding set of criteria for effective crowd collaboration support. We conduct a content analysis of ten real world projects to gain insights on their collaboration support features and collaborative interactions of crowd workers. This paper contributes to crowdsourcing research and practice by deriving recommendations for advancing the collaboration process design framework as well as for improving the conclusiveness of collaboration support on crowdsourcing platforms